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I. INTRODUCTION 


The purpose of this analysis is to discover whether the Leadership and 
Management Education and Training (LMET) of supervisory personnel in Navy 
commands is systematically related to measures of effectiveness (MOEs) such as 
exercise and inspection scores, Unit Status and Identity Report (UNITREP) combat 
readiness ratings, personnel retention, etc. The two focal questions of interest are: do 
increased proportions of LMET trained officers and enlisted supervisors relate to 
improved readiness and effectiveness; and does the length of time since having LMET 


“dilute” the effects of having had the course? 


A. ORGANIZATION OF THE THESIS 

The first chapter is an Introduction discussing the purpose of the thesis and 
describing the elements of the study. Chapter II briefly chronicles the historical 
aspects and development of LMET. Previous research in LMET and shipboard 
measures of effectiveness (MOEs) are discussed in Chapter III. The techniques used in 
data collection and analysis are the subjects of Chapter IV. In Chapter V, results are 
presented in the form of correlation coefficients between MOEs and 1) percentages of 
supervisory personnel onboard with LMET and 2) average length of time since 
personnel have been LMET trained. Chapter V also deals with the relative significance 
of the data. Chapter VI presents the authors’ interpretation of results and limitations 
in applving them. Conclusions and recommendations for improving the Navy's 


leadership program are discussed, as are recommendations for further research. 


B. DEFINITION OF LMET 

LMET is a formal, Navy specific training program designed to prepare 
supervisors and managers for leadership and management positions at key (threshold) 
points in their careers. LMET is based on research done by McBer and Company, of 
effective Navy leader behavior (as is discussed in Chapter II) and focuses on specific 
skills and individual initiative. LMET is now taught at 21 sites to about 30,000 Navv 
personnel each year. In fiscal year 1980 LMET cost the Navy about $17 mullion 
(Ref. 1] 


‘p. 207’ in 1986 the approximate cost of LMET was $21 million.! There are 19 
varieties of the course--each geared to the appropriate level in the chain of command 
(i.e. Leading Petty Officer, Senior Officer) and tailored to the warfare or staff 
community (surface, aviation, medical, supply) [Ref. 2: p. vi]. LMET uses lecture, case 
studies, role plaving, simulations, small group discussions, instrumented feedback (self 
assessment questionnaires), and individual written and reading assignments to convey 


to participants leadership competencies. [Ref. 2: Da] 


C. BASIS FOR RESEARCH 

Both authors of this thesis have attended LMET, LCDR Polley in 1981 and LT 
Cissell in 1982. Thev found it to be interesting and helpful to them. Their interest in 
researching this topic was encouraged by a request from NMPC-62 to analyze the 
relationship between LMET and unit effectiveness. 

An article published in avy Times earlier this year [Ref. 4], discussed a study 
sponsored by Naval Military Personnel Command (NMPC-62) Leadership and 
Command Effectiveness Division (Ref. 5] on what percent of their time Navy personnel 
spend in each of three areas: Technical Tasks, Leadership and Management. The 
study also asked its 983 respondents how well prepared they were for these tasks. The 
study found the individuals perceived themselves to be spending more time on 
leadership and management tasks than technical tasks as early as E-6. Findings also 
showed that a significant majority of respondents felt less prepared for leadership and 
management tasks than for technical tasks. An unnamed source in the Navy Times 
article said, “We're (the Navy's) just not spending enough time and energy on 
leadership.” “Only 21,000 enlisted persons a year--nine percent of the 240,000 in 
paverades E-5 through E-9--go through LMET. Considering turnover,...that figure is 
appallingly low.” 

[If LMET can be more directly tied to those criteria that are considered to be 
valid measures of effectiveness and readiness, then perhaps more effort will be made to 
ensure that more supervisors and managers receive the training before they enter a 
leadership position. If LMET does not relate to command effectiveness, then what 


specific areas of LMET should be targeted for reform? 


‘Information supplied by NMPC-62. 


se competency, as McBer defines it, is “a knowledge, skill, ability, motive, or 
other characteristic that can be demonstrated to relate directly to competent 
occupational performance” [Ref. 3]. 
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In 1986 about 7,500 officers and 20,800 enlisted personnel attended LMET.? 
Most attended through some training pipeline; that is, they went to the training as part 
of a string of schools scheduled between duty stations. LMET is required for all 
supervisory leadership personnel (E5 - O5) prior to assignment to: 

a} arduous sea duty, 

b) unaccompanied overseas shore duty, 

c) unaccompanied non-rotational sea duty, or 

d) neutral duty (neither sea nor shore duty). 
This requirement may be waived if 1) operational commitments override; 2) no quota Is 
available; 3) the member is going from one outus (outside continental United States) 
billet to another; or, 4) that person has already attended LMET.®* In addition to those 
attending LMET between duty assignments, about one-third of LMET seats are filled 
by TAD (temporary additional duty) students. These are people who come to LMET 
from a unit and then return to that same unit after the training. 

LMET, as it exists now, lasts one week (as in the case of Basic and Advanced 
Division Officer LMET), two weeks (as in Leading Petty Officer (LPO) and Leading 
Chief Petty Officer (LCPO) courses), or just two days (as in the case of the Command 
Effectiveness Seminar). LMET, in its present form, is best descnbed as an 
evolutionary form of the original LMET course designed in cooperation with McBer 
and Co. around 1977. Chapter II discusses the events leading to the development of 
LMET and why it was needed. 


3 Information supplied by NMPC-62. 
* Information supplied by NMPC-482. 
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Il. BACKGROUND 


A. MILITARY LEADERSHIP 

Good leadership is essential to the effectiveness of any organization. One finds, 
however, little agreement among scholars, researchers, or practitioners as to what 
leadership is, much less how to define good leadership. Definitions and theories range 
from those focusing on an individual’s personality and genetic traits to those describing 
leadership more as a process involving interaction between organizational purpose and 
individual behavior. 

Competent military leadership is essential to the effectiveness of each military 
unit as well as to the success of the U.S. Defense Department in accomplishing its 
goals. Military organizations have unique missions which often require humans to 
perform tasks which might otherwise be considered inappropriate, immoral, or even 
unlawful in any other setting. The military also may require submission to stricter 
rules, adverse environmental conditions, and any number of tasks contrary to personal 
preference. The men and women in the armed services are expected to perform ever 
more diverse and demanding tasks with existing or often fewer resources. The future 
role of the Navy as well as that of other services will place increasing pressure on 
mulitary leaders to do more and better with less. The shape of the future, because it 
points to increased technology, automation, and reduced manning levels, only sharpens 
the need for Navy officers and senior NCOs to acquire requisite leadership and 
management skills. 

In seeking those skills encompassed by leadership and management, one must 
first understand the concepts: how do kev people in the Navy define leadership and 
management? In a June 1987 conference on leadership held at the United States Naval 


Academy> 


VADM William Rowden, Commander Naval Sea Systems Command, 
distinguished leadership from management by defining leadership as “the ability to 
motivate people” and management as “a process of getting things done” [Ref. 6: p. 3]. 


Also at the conference, Professor Ben Schneider, University of Maryland, made a 


“On June 10-12, 1987, the Naval Academy and Navy Personnel Research and 
Development Center (NPRDC) co-sponsored a conference on leadership. There were 
90 participants and 35 speakers. Twelve active and retired flag officers attended 
included ADM Trost, Chief of Naval Operations. Several academic researchers in the 
leadership field spoke as did the Master Chief Petty Officer of the Navy. 


ie 


simular distinction: leadership is “energizing and directing others” and management is 
“a process of getting things done” [Ref. 6: p. 3]. Admiral Rowden also said that 
management is more easily learned than leadership. Perhaps this is because there is 
greater agreement over what management is while leadership still exists in a haze of 


theory and disagreement. 


B. HISTORY OF NAVY LEADERSHIP TRAINING 

Since World War II, the Navv has focused most of its training efforts on the 
individual as he. she first enters the service. Some highly technical ratings/designators 
require that the service member attend as much as eighteen months of classroom and 
practical training before being assigned to his.her first experience tour. Leadership and 
Human Resources Management topics are typically included in Academy curricula, 
recruit training, Officer Candidate School classes and at other “source” schools. 
Leadership training did not, however, receive serious consideration until the 1950's 
when symptoms such as a proportionally large brig population prompted action by the 
Secretary of the Navv to “shore up” leadership deficiencies [Ref. I: p. 197}. In 1958, 
General Order 21 was issued. It defined leadership® and ordered Commanding Officers 
to incorporate leadership training into their command training plans. With minimal 
real change resulting from the order, it was re-issued in 1963 but again was ineffective 
perhaps due to the lack of assistance or specific guidance given to commanding officers 
[Ref. 1: p. 198]. In 1966, leadership training was incorporated into General Militarv 
Training (GMT). Each sailor was to receive ten hours per year in leadership style, 
chain of command, authority, responsibility, and accountability. ’ With standard lesson 
plans and material, leadership training was scheduled five times annually along with 
venereal disease prevention and blood donorship. 

In 1970, then Chief of Naval Operations (CNO), Admiral Zumwalt, laid the 
groundwork for the Human Goals Program, later to be called Human Resources 
Mlanagement Support System, which encompassed leadership training, racial 


awareness, drug and alcohol counseling, and overseas diplomacy. A new two-week 


© Leadership is the art of accomplishing the Navy's mission through people. It is 
the sum of those qualities of intellect, of human understanding and of moral character 
that enable a man to inspire and to manage a group of people successfully. Effective 
leadership, therefore, is based on personal example, good management practices, and 
moral responsibility. [Ref. 7] 


’ With the exception of leadership stvle, these topics are now taught in an eight 
hour command level workshop: “Military Rights and Responsibilities.” 
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formal, external (to the command), Leadership and Management Training (LMT) 
course represented the leadership portion of the new program. It was authorized to be 
taught at fifteen sites, but the demand for quotas exceeded the supply and “bootleg” 
versions of the course were created (over 150 unauthorized courses) [Ref. 8]. 

In August 1974, shortly after he relieved ADM Zumwalt as CNO, ADM 
Holloway ordered a review of all officer and enlisted leadership and management 
training. The Chief of Naval Education and Training (CNET) convened a panel 
headed by CAPT Carl Auel (Chaplain Corps) assisted by Fred Fiedler (a scholar in the 
leadership field). Over a three month period, the panel examined earlier and existing 
leadership training and proposed a method for designing an “ideal” training model. 
Their report refered to development of a system, not a course, implying that much 
more than a single course would be necessary to correct the leadership training 
program. “Without an LMET system, the first phase of which is a clear and 
comprehensive definition of requirements by line managers, any further expansion, 
consolidation, or reprogrammung of current training efforts would meet fleet needs at 
the levelay chance: Siivciee so. 

It was in 1975 that McBer and Co. became involved in the Navy Human 
Resources Management Program. McBer, a Boston-based consulting firm established 
by Dr. David C. McClelland and David Berlew in 1970 [Ref. 10: pp. 35 & 39], was 
contracted to improve the effectiveness of Human Resource Management (HRM) 
Specialists. McClelland, a Harvard psychologist, had focused much of his work on 
improving the screening process for hiring employees. He found that in many 
Organizations, the tests they were using to screen applicants tested for academic 
potential rather than for skills that would be reflected in job proficiency. McClelland 
believes that people should be hired and trained based upon competencies. 
“Competencies are not aspects of the job but characteristics of the people who do their | 
job best.” [Ref. 10: p. 40] ye 

After identifying what behaviors superior HRM specialists demonstrate better or 
more often than average specialists do, McBer devised a training model based on 
“competencies” [Ref. 11]. McClelland’s theories on competencies and how they relate 
to achievement are explained in further detail in The Achieving Society [Ref. 12] and 


“Testing for Competence Rather Than for ‘Intelligence’ {Ref. 13]. 


McBer’s approach--to sample (using Behavioral Event Interviews)® high 
performers and average performers to train people to do those things that separate 
high performers from their peers--had both scientific and practical appeal to the Navy. 
In January 1976, after abandoning internal efforts to develop a new leadership training 
program, and under high level pressure to produce tangible results, several civilian 
contractors Were asked for proposals. The unconventional approach of McBer was 
selected. Using the same technique employed in the Navv HRM Project, McBer 
analyzed the results of interviews with Navy supervisory personnel previously 
categorized by thier commanding officers as either superior or average leaders. 
[Ref. 1: pp. 204 & 205] In 1976, McBer began sampling Navy Leading Petty Officers, 
Chief Petty Officers, and Commissioned Officers first on the West Coast and then on 
the East Coast. Their first model included twenty-seven competencies. In 1978 and 
1979, pilot courses were taught by Navy instructors and evaluated by System 
Development Corporation [Ref. 16]. Evidently, these early courses were based on all 
twenty-seven competencies. To validate their findings, McBer later sampled 1,000 
Navy officers and enlisted personnel using nine tests to measure competency elements. 
Behavioral Event Interviews were also conducted on a subset of 100 testees. Sixteen of 
the original twenty-seven competencies were found to be significantly related to 
superior leadership in the validation phase. These sixteen competencies, listed in Table 
|, are now the backbone of most of the current LMET courses. 

The premise behind LMET is that the sixteen competencies can be learned, and 
increased use of the competencies will lead to better leadership and management and 
hence improved effectiveness. LMET competencies are acquired through a five-step 
process: 


1. Recognition (identifying knowledge, skills, values, etc. present in 
cases, incidents) 


tJ 


Understanding (integration and connection with one’s own experience) 


3. Self Assessment in Relation to the Competency (discovery of one’s own level in 
each competency and identification of areas for specific improvement) 


4. Skill Acquisition and Practice (practical exercises/classroom applications) 


’ Behavioral Event Interviews are similar in technique to the Critical Incident 
Interview developed by Flannagan during WWII [Ref. 14]. Each subject is asked to 
describe in detail unsuccessful and successful events in his,her career. Thorough 
probing by the interviewer leads to a clear account of what lead to the event, who 
participated, and the interviewee’s behavior and feelings in that situation. [Ref. 15] 
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5. Job Application (classroom feedback and identification of goals, action steps, 
and obstacles on one’s next job) (Ref. 15]. 


Initially LMET was taught at five levels: 1) Commanding and Executive 
Officers, 2) Department Heads, 3) Division Officers, 4) Leading Chief Petty Officers 
(LCPOs), and 5) Leading Petty Officers and at two sites: Little Creek, VA and 
Coronado, CA. Since then, LMET sites have expanded to 21 locations. LMET has 
since been tailored to the specific needs of each of the warfare communities as well as 
to staff commuunities such as Navy Medical Command (NAVMED). [Ref. 2: pp. v1 
and 43-48] 

In 1983, a resurvey by McBer consultants essentially validated the “Fleet 


er 
/ 


Competency Model” with some modification on earlier competencies. Improvements 
in their methodology also allowed specification of the tvpe of situation a given “critical 
incident” was observed in. This allowed training to focus on how one might deal with 


a variety of situations, 1.e., when to employ which competencies. [Ref. 2: pp. 50-51] 


16 


More recently, the Command Effectiveness Study” results have had an Impact on 
LMET content, particularly the courses for Prospective Commanding and Executive 
Officers (now replaced by the Command Effectiveness Seminar) and the LCPO course. 
Essentially LMET has been shaped not only by the initial research done by McBer, but 
also by feedback from participants, evaluations made by observers, and subsequent 
research. 

LMET is now the approved method for Naval leadership and management 
training. The course components have not, however, been systematically included in 
Naval Academy curricula, Officer Candidate School classes, or Navy Reserve Officer 


Training Corps (NROTC) requirements. 


"The results of the Command Effectiveness Study are discussed in Chapter III. 


le 


HI. LITERATURE REVIEW 


In a memorandum for the record by Naval Miltary Personnel Command, 
NMPC-6, dated December 7, 1978, an evaluation plan for LMET was laid out in four 
phases: 

I) Atteview or E ViET tomar 


2. Course validation 
3. LMET delivery to the Navy 
4. LMET evaluation 


Phase four called for a longitudinal evaluation of the impact of LMET on both 
individuals and their organizations. While several efforts have been made to discover 
the impact on individuals (as discussed in the following section) no evidence was found 
that any thorough study has been done to discover the tmpact LMET has had on 


organizational effectiveness. 


A. PREVIOUS EVALUATIONS OF LMET 
1. NPRDC/ONR/NPGS (1977) 


McBer’s initial work in identifving competencies and designing the training 
was scrutimized by several professionals among whom were Dr. R. F. Morrison of 
Navy Personnel Research and Development Center (NPRDC), Dr. B. T. King of the 
Office of Naval Research (ONR), and Dr.s C. K. Eoyang and R. S. Elster of the Naval 
Postgraduate School (NPGS). Some of their concerns were: 


¢ avery small, unrepresentative sample (n= 36) in developing the 27 competencies 
(King and Elster) 

e reliance on supervisor evaluations and behavioral event interview results (rather 
than including direct observation and peer and subordinate perceptions) (King 
and Eoyang) 

¢ inappropriate coding techniques (1 = attribute present regardless of how often; 
QO = attribute not present, 1.e. wasn’t brought up in the interview) (Morrison) 

e shaky statistical techniques: 


* multiple regression used to “predict” superior performance ratings using 27 
competencies (independent variables) for such a small sample (King) 


* — multicolinearitv of variables (Eoyang) 
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* factor analysis using two few cases (Morrison) 


* significance testing at the .10 level (King)!° 
After such sharp criticisms, it is surprising that LMET ever got off the ground. 
[t would appear that McBer’s competencies, however arrived at, intuitively appealed to 
reviewers and are similar to other characteristics of successful leaders such as those 
found in the Handbook of Leadership. by Stogdill.!! 


2. SDC (1979) 

Between May 1978 and May 1979, System Development Corporation (SDC) 
evaluated LMET pilot courses for Leading Petty Officers (LPOs), Leading Chief Petty 
Officers (LCPOs), Prospective Commanding and Executive Officers (PCO/PXO) and 
je vileieeiistmactorns. Objectives of the assessment were: (1) to perform on-site 
evaluations of the delivery of the courses; (2) to review instructor guides and student 
journals; and (3) to provide specific recommendations for management decisions 
concerning the assignment of Navy instructors to deliver the courses. 

The SDC assessment was not intended to measure impact of the training on 
subsequent performance, but did attempt to discover whether students were receptive 
to the course material and absorbing any of it. A sample of SDC’s findings: 

e Navy instructors were in need of training in facilitation techniques. 


e Course materials needed to be “de-civilianized”, that 1s, made more suited to 
nulitary needs and situations. 


e Participants enjoved the courses. 
e Time boundaries limited the ability to use very many practical exercises. 


e SDC recommended courses be standardized and offered to all targeted levels of 
Navy personnel. [Ref. 16] 


It is interesting to note that SDC also participated with McBer in the data 
gathering phase of the LMET project [Ref. 3}. Their expertise in LMET design 1s 
useful; however, one might question their complete objectivity as they may have had 


some stake in the success of LMET. 


- Excerpted from memorandums written by Dr. Morrison, Dr. King, Dr. Elster, 
and Dr. Eoyang in response to requests to review McBer’s research. [These memos 
were supplied by Dr. Carson K. Eoyang, PERSEREC, from his personal files. 


''These characteristics include but are not limited to: intelligence, energy, 
judgement decisiveness, integritv, achievement drive, dominance, drive for 
responsibility, initiative, sociability, assertiveness, emotional balance and control, and 
ability to enlist cooperation. [Ref. 18: pp. 74-75]. 
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3. Davies (1980) 

In his thesis Davies discussed the need for the Army to evaluate its leadership 
training. He presents an extensive review of the leadership theories contributing to the 
Army’s organizational leadership model, their training programs, and the leadership 
training of the other services. 

In his discussion of the development of the Navy’s LMET program, Davies 
traces the evolution from the early 1970’s when the Navy had 157 different leadership 
courses through the research by the McBer Company which identified the sixteen 
competencies which form the basis of the current LMET courses. He also notes that 
there has been no formal evaluation of LMET, although at the time of his work (1980) 
the Chief of Naval Education and Training (CNET) was “progressing toward an 
internal evaluation plan to determine whether the course 1s actually teaching what it 
was designed to teach.” [Ref. 19] 

As discussed by Davies and elsewhere in this thesis, the evaluation of LMET 
is complicated by the lack of a control group within the Navy. This is because the 
Navy has adopted the policy of not denving LMET training to anv personnel for the 
purpose of establishing a control group. 

In presenting his recommendations, Davies proposed two separate plans, the 
organizational leadership training evaluation plan and the Army’s Leadership and 
Management Development Course evaluation plan. Within each of these areas he 
offered specific objectives and steps to measure the achievement of the objectives. One 
of his lower echelon divisions is organizational performance for which the stated 
objective is: to determine if the leadership training program is reflected in changes in 
the operational performance of the units to which the newly trained leaders are 
assigned. Although this objective roughly parallels the purpose of this thesis, there are 
numerous and significant differences between the Army's program and the Navy's 
LMET program, thus precluding further development along the path which Davies has 
laid out. For example, the Army takes a decentralized approach for its Leadership and 
Management Development Course, allowing these experience based workshops to flow 
according to the needs and backgrounds of the individuals attending. Whereas the 
Navy has adopted a strict, centrally controlled format for its LMET courses. 

In summary, Davies achieves his stated purpose of raising the question of 
evaluation of the Army’s leadership program and of offering a plan whereby the issue 


of training effectiveness can be studied. That such an ambitious evaluation scheme as 
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he proposes will ever be undertaken by the Army or any other agency is questionable 
because such an evaluation would require a high level of commitment including several 


million dollars for the research effort alone. 


4. Parker (1981) 

Donald F. Parker is a retired Navy Captain and (in 1981 when this reference 
was published) was Assistant Professor of Organizational Behavior and Industrial 
Relations in the Graduate School of Business Administration at the University of 
Michigan. Immediately preceding his retirement from the Navv he was CO of the 
Navv Personnel Research and Development Center. In a chapter he wrote for Military 
Leadership (Ref. 1], Parker reviews the events leading up to LMET development, the 
research upon which LMET 1s based, and LMET course design and deliverv. 

The following are some of the findings and conclusions made by Parker: 

e LMET could have been developed with internal expertise. 


¢ LMET was not designed with a clear comprehensive definition of requirements 
as Was recommended by a panel headed by Chaplain Auel [Ref. 9]. 


e LMET was not developed under the Interservice Procedures for Instructional 
Systems Development (ISD) which Parker believes led to inadequate learning 
objectives; inconsistencies between tests, instruction material, and stated 
objectives; and difficulty in measuring the success of the training. 


e Analysis for LMET design was “deficient with respect to concept definition, 
research design, data collection, data analysis and interpretation” [Ref. 1: p 198]. 


e¢ LMET courses don’t include the concept of contingency, 1e., how to select 
appropriate behaviors in differing situations. 


From his observations of classroom training, Parker found that the flow from 
lecture to discussion to small group activities helped to maintain student interest and 
provided frequent opportunities for students to express their opinions and trade ideas 
with peers. He found that in practice LMET instruction differed somewhat from class 
to class and location to location as instructors sought to motivate each group. He also 


found that the course was well accepted by students. [Ref. 1: pp. 207-208] 


5. Vandover and Villarosa (1981) 

In 1981 Vandover and Villarosa, two Naval Postgraduate School students, 
interviewed a cross section of 51 LMET graduates and their immediate supervisors and 
subordinates from 13 operational commands. In their pilot studv for evaluation they 
sought to discover any improvements over non-graduates in the knowledge or behavior 
of LMET graduates. They found no systematic link between LMET and leadership 


related behavior changes. However, some of the trends they discovered include: 
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e Seniority of graduates appeared to negatively correlate with behavior change. 


e Evidence of behavioral changes as a result of attending LMET were greatest 
among Leading Petty Officers (LPO’s) and lowest among Chief Petty Officers 
(CPO’s). . 


e People who were marginal performers before the training, experienced more 
impact from the course. 


e Knowledge’familiaritvy with LMET competencies seems to deteriorate after as 
little as six months following training. 


e Graduates’ perceptions of their own improvement after training were not 
validated by their supervisors and subordinates. 


e Graduates who were members of high-performing, effective units were more 
likely to have shown behavioral improvements after training. [Ref. 20} 
6. Abe and Babylon (1982) 

Using McBer’s Behavioral Event Interview technique, Abe and Babvlon, two 
Naval Postgraduate School students ”...sought to find if the specific competency of 
delegation is more often demonstrated by superior Navy personnel and if LMET 
training has any significant impact upon managerial effectiveness and the use of 
delegation’. 

They found no relationship between LMET attendance and use of delegation. 
Superior performers did not use delegation more than average performers. LMET 
graduates were equally distributed among superior and average performers. One 
statement they made was particularly astounding: “The fact that delegation is taught 
in LMET but not used in daily performance could also be used to suggest that the 
Navy is an unfavorable environment in which to practice the skills learned in LMET” 
cian oul: 

Perhaps LMET’s design by civilian consultants, who had very little Navy 
specific experience, has resulted in training unsuited for the Navy environment (or as 


LCDR Foley suggests [Ref. 22], unsupported by unit effectiveness). 


7. Foley (1983) 

Another Naval Postgraduate School student, LCDR Patricia Foley, 
interviewed 70 LMET graduates seeking to discover what incentives and constraints 
affect the utilization of LMET competencies. She found no statistical differences 
between LMET graduates and the control group. Through her interviews, she found 
the following organizational factors influenced the use of competencies by graduates: 

e Time constraints 


e Manning constraints 


a 


Leadership example as set by superior 
Communications flow 

Attitude towards inspections (short sightedness) 
Lack of emphasis on subordinate development 
No support by senior members of the command 


Lack of a reward svstem for competency use 


Those who had demonstrated behavioral changes that they attributed to LMET 
exhibited these characteristics: 


A strong desire to change their behavior 

Felt they had room for improvement in leadership and management 
Were more likely to be junior with some leadership experience 
Returned directly to management positions after graduating 

Had some initial success in practicing the competencies 

An immediate superior or peer had served as good role model 


They were more likely to be assigned to a command noted for its organizational 
effectiveness and that stresses subordinate development 


She recommended that LMET be continued and reinforced at the unit level 


and that, through the HRM program, commands improve communication, stress 


subordinate development, and improve problem solving techniques. [Ref. 22] 


8. Glenn (1987) 


Mike Glenn, a former Navy Organizational Effectiveness Consultant presently 


working at Naval Training Systems Center in Norfolk, VA, is finishing a doctoral 


dissertation on “Senior Management Perceptions of Actions to Support Post-Training 


Ltilization of (LMET)”. Of specific concern to Mr. Glenn are the following: 


Se 
xe 


Which management and supervisory actions in support of Job Linkage and 
Follow-up!* do senior Naval managers perceive to be important for their 
subordinate managers/supervisors? 


To what extent do senior Naval managers perceive that the important 
management and supervisory actions in support of Job Linkage and Follow-up 


are practiced in their organizations by their subordinate managers, supervisors. 
Picieee oy 


l2 Job Linkage and Follow-up refer to specific activities in this dissertation. Job 


Linkage, as Glenn defines it, is re-entry of the trainee into the workplace. He defines 
Foliow-up as on-going support of learned behaviors on the job. 
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He surveyed 106 Navy operational commands asking senior managers to rate 
the importance of various actions which may be taken by management to support the 
full use by personnel, on the job, of behaviors learned in LMET. He also asked them 
to indicate whether they had ever observed each action in any organization and in their 
present command. So far he has found that senior Navy managers believe assignment 
of a role model (supervisor or co-worker), trainee goal setting for job performance, and 


trainee environment are important to Job Linkage. 


9. LMET Sites 
At least two LMET sites are gathering data from Commanding Officers (COs) 
of Temporary Additional Duty (TAD) attendees about six months after graduation.!? 
Their purpose 1s to gauge whether the COs are pleased with the “results” of LMET. 
Their questions include: 


e Has the individual’s leadership and management performance improved after 
completing LMET? 


e What improvements in performance if any have been seen since attending 
bey) 


e What improvements have you seen in the work group? 

One site found that about 64 percent of COs responding noticed an 
improvement in the performance of graduates. The sample size is very small thus far 
(n= 18), but the effort shows considerable promise in specifying what LMET does for 
graduates and their commands. 


~. 10. Command Effectiveness Study (1985) 

/ Although not specifically related to LMET, this study, done bv McBer and 
Co., turned from a focus on individual performance to characteristics distinguishing 
superior from average commands. After identifying criteria/indicators of superior 
command performance, a sample of outstanding and average operational commands 
were observed, interviewed and surveyed. 

Four survey instruments were used in the study: 


¢ Navy Competency Assessment Profile (NCAP) - asked respondents to rate 
themselves on the sixteen fleet competencies and whether they had attended 
avi eA. 


‘3 Correspondence with LMET sites in Little Creek, VA and Mayport, FL, 
revealed that earlier this vear they began sending questionnaires to Commanding 
Officer's of TAD attendees about six months after they graduated. 
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¢ Command Information Questionnaire (CIQ) - replaced the NCAP in the second 
phase of the study (84-85); asked people to rate their command on the 
characteristics of superior commands identified in the first phase of the study 
(1983) 


¢ Systematic Multiple Level Observation of Groups (SYMLOG) - designed to 
identify the dynamics of work groups in terms of the roles assumed by 
individual group members 


¢ Commutment Index - used as a supplement to the SYMLOG; assessed the level 
of commitment personnel felt to their job and command 


The reason given for replacement of the NCAP in the second phase of the 
study was: “Although relevant to competencies, the NCAP results were not useful in 
understanding differences between superior and average commands” [Ref. 25]. 

In a pilot test of the NCAP in 1983, 100 Navy enlisted personnel answered the 


questionnaire. The conclusion was: 


Although the overall competency rating scores did not differ significantly as a 
function of LMET training, LMET-trained individuals were better able to 
differentiate their abilities in using the competencies. Persons without LMET 
training tended to rate themselves similarly on all 16 competencies. [Ref. 2: p. 
50] 

Despite the drawback of using a self assessment instrument such as the NCAP 
to measure the effect of LMET; these statements are supported by some of the studies 
discussed previously. 

The primary product of the Command Effectiveness Study was a model of an 
organizational system whose “parts” are interrelated. These components have certain 
characteristics which distinguish the superior organization from an average one. The 
model consists of three areas which are further broken down into thirteen levels: 

ee eople 
- CO 
- XO 
- Wardroom 
- Chiefs Quarters 
- Crew 
e Relationships 
- CO-XO 
- Chain of Command 
-  Exiernal 


e Activities 


= 


- Planning 

- Maintaining Standards 

- Communicating 

- Building Esprit de Corps 

- Training and Development 

The results and lessons learned through the Command Effectiveness Study 

(CES) are the basis for the Command Effectiveness Seminar (also Known as the 
Command Excellence Seminar). CES results have also been incorporated into most of 
the LMET courses. [Refs. 24,25} 


11. Command Excellence Seminar Feedback Analysis (1987) 

This Caliber Associates report summarizes an analysis of feedback (course 
critique) sheets filled out by 215 Command Excellence Seminar attendees. The 
objective of the report was to identify results related to improved mission readiness 
experienced by seminar attendees by conducting a content analysis of two items from 
the feedback sheets. These items asked the respondent to give examples where the 
seminar helped them or their commands do the job better. 

The responses were fairly homogeneous in that virtually everyone reported 
some type of improved performance in, or greater awareness of some dimension of 
leadership. Overall, the report indicated that the Command Excellence Seminar is 
beneficial. Participants responded enthusiastically and attribute considerable personal 
success to the course. The data did not support anv specific conclusions regarding 
“outcomes” in the form of organizational impact from the course. This was due in part 
to the lack of specificity in the feedback questions and exploratory nature of the 
analysis. [Ref. 26] 


12. Summary 
What can be learned about the effectiveness of LMET from the studies 
presented thus far? Training evaluation is often categorized into four types of 
measures: 
1) Participant Reactions 
De erewidence of Cearmne 
3) Evidence of Behavioral Change and 


) Results in Operations (impact on organizational effectiveness). [Ref. 27] 


aE 


a. Participant Reactions 
In reading the studies discussed earlier and in corresponding with former 
and current LMET instructors we found that students generally react positively to 
LMET. Many have stated that they wish they could have attended earlier in their 


Careers. 


b. Evidence of Learning 
Pre and post-testing of students by SDC during their assessments of pilot 
courses indicated that students were gaining expected knowledge levels. But Vandover 
and Villarosa’s [Ref. 20: p. 87] observation regarding deterioration of knowledge after 
as little as six months leads one to conclude that much of the material learned is short- 
lived. 
c. Evidence of Behavioral Change 
This was the primary focus of three of the Naval Postgraduate School 
theses (Refs. 20,21,22]. None of them found a systematic link between LMET 
attendance and improvement in leadership behavior. They were able to discover some 
individual and organizational factors that intervene in behavioral changes. Glenn's 
approach in his dissertation is to discover these factors through the perceptions of 


senior managers [Ref. 23]. 


d. Results in Operations 
As stated earlier, this ground is yet uncovered in LMET evaluation. There are a 
number of reasons why such research has not been done yet, including: 


e Lack of a “control group” (a command or unit completely unaffected by 
vie |) 


¢ No baseline research on effectiveness levels prior to LMET implementation 
e Manv uncontrolled intervening variables 


e Instability of measures of effectiveness (MOEs). 


B. STUDIES ON MEASURES OF EFFECTIVENESS 

How then can LMET be evaluated as to its effect on mission effectiveness and 
readiness? The first step is to select criteria that reasonably represent mission 
capability. Three studies in this area are presented from which several candidate 


Gependent variables have been gleaned. 


pee 


1. Horowitz (1986) 


Dr. Horowitz found at least fifteen studies identifving quantitative links 


between Manpower, Personnel, and Training (MPT) factors and unit performance. 


Several of these studies address the payoff to training. He suggests such measures as: 


Operational Propulsion Plant Examinations (OPPEs) 
Operational Readiness Examinations(OREs) 
Selected Exercises (including live firing exercises) 
Excellence Awards (such as Battle Efficiency “E”) 
Bombing scores (for aviation units) 

Air Combat Maneuvering Ranges 

Simulator performance 

Casualty Reports (CASREPs) 

Unit Status and Identifv Reports (UNITREPs) 
Board of Inspection and Survey (INSURY) 


Maintenance and Material Management (3M) 


Dr. Horowitz found shortcomings among most of the measures yet he did not see them 


as insurmountable barriers to research in the area of relating MPT factors and unit 


MOE’s. For example: 


Training exercises such as OREs and OPPEs are prepared for, and would 
therefore reflect an “upper bound” on performance. 


For selected exercises failing grades may not be numerically recorded. 
Only one ship per squadron can be awarded the Battle “E”. 


CASREPs and UNITREPs are self-reported, not objective and criteria vary 
widely among commands. 


3M data suffers from reporting errors and differences from ship class to ship 
class and over time. [Ref. 28} 


2. Davilli and Schenzel (1986) 


Davilli and Schenzel, two Naval Postgraduate School students, used Refresher 


Training (REFTRA) ORE and Battle Problem scores as dependent variables in creating 


Multiple Regression Models of the relationship between readiness and a number of 


manpower, training, and other evaluative measures. They used a small sample of ships 


(n= 44), however, and had to obtain much of the data by physically searching through 
REFTRA files in Guantanamo Bay, Cuba. 
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Assuming ORE is a universally accepted measure of readiness, Davilli and 
Schenzel’s results indicate a multivariate approach to predicting readiness is feasible. 
The variables with the greatest predicting power (Beta coeficient and significance level) 
in their model were billet vacancies at 90 days and 180 days prior to ORE. Variables 
such as average drill periodicity, average school qualification, and average watch 
qualification 30 days prior to ORE were poor predictors (low significance and Beta 


coefficients, and (for one variable) the opposite sign than was expected). [Ref. 29] 


3, Chatfield and Morrison (1987) 

Researchers at NPRDC assessed the consistency and stability of 20 surface 
ship measures of effectiveness from fiscal vear 1982 to 1984 [Ref. 30]. Their purpose 
was to create a pre-change baseline that could be used later in evaluating the effect of 
the new Surface Warfare Officer (SWO) career path on readiness and performance. 
They proposed using a multiple measure approach, assuming that no single measure 
was an appropriate evaluation standard. The unit measures they looked at included: 

¢ PEB (Propulsion Examining Board) 

e PMS (Preventive Maintenance System) 

¢ NWTI (Nuclear Weapons Technical Inspection) 
¢ REFTRA (Refresher Training) Quick Look 

e Post-TRE (Training Readiness Evaluation) 

¢ CASREPs (Casualty Reports) 

e Personnel Retention 

e TRA (Training Readiness Assessment) 

¢ CSRT (Combat Svstem Readiness Test) 

e Safety Inspection 

¢ Command Inspection (done by immediate superior command) 
e INSURYV 

e ARE (Aviation Readiness Evaluation) 

e Battle “E” competition 

e PQS (Personnel Qualification Svstem) 

* USUTREP 

Chatfield and Morrison found that data on ship performance had poor year to 
vear stability and was inconsistent among different MOEs for the same ship. They 
concluded that the measures were too unstable to use as a baseline for evaluating 


policy revisions. They did not recommend collection or analysis of other measures as it 


Z 


would likely lead to similar results. Instead Chatfield and Morrison recommended 


review and revision of MOEs to improve their reliability and validity. 
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IV. METHODOLOGY 


A. SELECTION OF VARIABLES 

After reviewing the studies cited in chapter III and phone conversations with 
tvpe commander staff members about what the admirals look at in determining which 
ships get the Battle “E”, the following measures of shipboard effectiveness were 
Sclectcu: 


¢ Percent (of an eighteen month cycle) time spent in each of four C-ratings!* as 
repomec one NIT REPRSs: 


¢ REFTRA scores!> 
¢ 3M PMS inspection scores 
* /@RERE scores 


¢ Supply Management Inspection (SMI) results!® 


¢ Personnel retention rates!’ 


Because a secondary goal of this study was to create a framework for future 
evaluation of LMET, variables were selected not only on the basis of face validity but 
also on data availability and potential for quantification. The objective was to obtain 


data on as many ships as possible through fairly routine reports and records. 


'SC-rating is an overall status based on a composite of the unit’s readiness in 
four resource areas: 1) equipment and supplies on hand, 2) equipment condition, 3) 
personnel, and 4) training. A rating of Cl means the unit 1s fully combat ready, C2 - 
substantially combat ready, C3 - marginally combat ready, C4 - not combat ready 
(Ref. 31]. Ships which spent more than half the cycle in programmed overhaul or 
conversion (C5) were not included. Anv time the remaining ships spent in C5 was also 
subtracted from the total before calculating the percentages. 


'> Because verbal (ordinal) scores were supplied by type commanders these were 
converted to equivalent (though somewhat arbitrary) numerical scores, Le., 
unsatisfactory - 0, satisfactory - 1, outstanding - 2. 


se Supply Management Inspection results were also provided in verbal (ordinal) 
terms and were converted in the same manner as REFTRA data: from a series of 
verbal scores in certain inspection areas. On the advice of a former SMI team 
member, the areas used to create an overall SMI score were: supply support, food 
service, retail operations, and aviation supply support (for those ships with a separate 
division for this purpose). 


17\et retention rates over the entire 18 month cycle. Net rate equals the number 
who reenlisted or extended divided by the total number eligible for reenlistment. The 
rates were given for first term, second term, and career. 


sy 
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The next logical step in the analysis was to measure the training itself. Since 
there was no control group or baseline measure, the next best alternative was to 
quantify LMET’s existence in each unit. To do this one must know: 


e Who was onboard each ship (a minimum of six of the eighteen months) in a 
supervisory or management position (E5 or above)? 


e Who, of the supervisors;managers, has had LMET and when did they attend? 
From this information the following variables were gleaned: 


e Percent of enlisted supervisors onboard for at least six months who had been to 
Evi at leastzonce 


e Percent of officer personnel onboard for at least six months who had been to 
LMET at least once 


¢ Percent of both officer and enlisted supervisors who had attended LMET 
e Average number of years since enlisted supervisors had attended LMET?® 
e Average number of years since officer personnel had attended LMET 


e Average number of years since both officer and enlisted supervisors had 
attended ie.yi ah 


Appendix A lists all variables used and the mean, minimum, maximum, and number of 


ships reporting for each variable. 


B. SCOPE 

The sample consisted of 285 surface ships. This was all surface ships under 
Commander Naval Surface Forces, Atlantic and Pacific, less those who were in 
overhaul or other programmed repair more than half of the eighteen month cvcle 
(January 1985 - June 1986). The eighteen month period chosen 1s the latest complete 
competitive cycle. 

Since MOEs differ so much among surface, aviation, submarine, and shore 
activities, it was decided that the analysis should be limited to only the surface 


community in this preliminary study. 


'8Since the first LMET courses were taught in 1978, for those who had never 
attended, their “vears since attendance” was set at eight years. The maximum number 
of years since attending could not be greater than seven (1985 - 1978) for those who 
had attended. Eight years was chosen so as not to give non-attendance too much 
weight in this variable. 


C. DATA SOURCES 

Chief of Naval Operations (OP-64) supplied a tape of C-ratings and 
corresponding dates for all surface ships.!? The remaining data on MOEs were 
collected from type commanders (Commander Naval Surface Forces, Atlantic and 
Pacific).-2 A list of Unit Identification Codes (LICs) was supplied to Defense 
Manpower Data Center (DMDC) which in turn provided a tape containing social 
securitv numbers of all E5s and above onboard these ships who show up on at least 
three quarterly manpower reports. This tape was forwarded to Naval Education and 
Training Command (CNET) in Pensacola, FL where the social security numbers were 
matched with Navy Integrated Training Resources Administrative System (NITRAS) 
files of LM[ET graduates providing data on who had attended LMET and when they 
attended. Courses numbers and titles supplied to CNET are listed in Appendix B. 
DM DC cleansed the data of cases in which an individual had attended a school which 
did not include LMET during the time he/she attended and grouped the data by unit 
yielding the LMET variables discussed earlier. Further information regarding the data 
received from NITRAS is included in Appendices C and D. 


D. ANALYSIS TECHNIQUE 

Because so many variables affecting each ship’s readiness and performance are 
unknown or unavailable, statistical techniques and conclusions are limited to describing 
hypothetical association between LMET and effectiveness. To estimate the degree of 
association between LMET variables and MOE variables, the Spearman Coefficient of 
Rank Correlation (Ps) was computed. Like other measures of correlation, Pp, varies 
from -1 (a perfect negative relationship between two variables) to +1 (a perfect 
positive relationship between two variables). This non-parametric test was chosen 
because of the ordinal nature of much of our data and the unsuitability of the data for 
more classical procedures (randomness, normal distribution, independent observations, 
etc.). [Ref. 32] 

This method of analysis does not lend itself to profound conclusions regarding 
LMET’s effect on unit performance or combat readiness. However this preliminary 
study is considered to be the first step in designing and testing a model for LMET 


evaluation against unit effectiveness criteria. 


\? The information obtained on C-ratings was classified “Secret”. 


*OMfuch of the data obtained from the type commanders were classified 
“confidential”. 


V. RESULTS 


Results of Spearman’s Rho Rank Correlation tests are shown in Tables 2 and 3. 
They are separated by fleet because of the often extreme differences in scoring, 
reporting, and standards between the fleets. In fact correlation between many of the 
Variables and fleet was significantly high (p, ranged from .0009 to .4309). Eight out of 
eighteen were significant at the @ = .05 level. 

In interpreting the results of this correlation, as listed in Tables 2 and 3. one 
must use caution. First, what do the numbers themselves mean? 

The top number ts the coefficient of correlation (Ps) which ranges from negative 
one (-1) to positive one (+1). This p, measures the strength of the relationship 
between the two variables adjacent to it in the matrix. A p, of negative one would 
indicate perfect negative correlation between the two variables, i.e., as one variable 
increases the other always decreases. While a p, of positive one would indicate that 
ships with high values in one variable also possess high values of the other variable. 
Such extreme values then indicate the strongest possible relationships between 
variables. A p, closer to zero (0) tends to show a weaker relationship between 
variables, with zero indicating no relationship between the variables. [Ref. 32: p 562] 

The second number in each of the “cells” of Tables 2 and 3 indicates the number 
of cases available to compute that particular p, value. This figure ranges from 24, in 
the case of the number of ships in the Pacific Fleet for which OPPE scores were 
available, to 153, the total number of ships in the study from the Atlantic Fleet. As 1s 
obvious in observing these figures, not all ships had scores or other measures available 
for this study. This variation in number of ships sampled for any given 
inspection/examination reflects differing operating schedules, availability of inspection 
teams, emergent fleet requirements (which occasionally require cancellation or 
rescheduling of inspections), and other factors bevond the scope of this study. 

The third and final statistic captured in each “cell” of the matrices in Tables 2 
and 3 1s significance level. This provides the reader an estimate of whether the 
relationship indicated by p, is probably due to chance or some systematic relationship 
between the two variables tested. Significance levels as shown in the tables indicate the 


likelihood that the relationship is merely due to chance, therefore values closer to zero 


34 


IRA BUE.2 


SPEARMAN’S RHO RANK CORRELATION COEFFICIENTS 
ee Ger CE Eh 


Percent enecn: fmetcent Average Average Average 
Enlisted Officers Both mi slnce Wt Sitec we SiMe 
alert: | weve ie we LMET \ PVE Eee 


ME 
Enlisted Officers Both 


“o lime _.0918 0202 VSS EalOZG -.0137 -.1323 
in C-] m= 153 N= 153 N= 153 N= 153 N= 153 N= 153 
Siglw2oe sig .SO4 Ss ie sig .207 sig .867 Sic2.h03 
Yo lime _.0143 -.0036 -.0466 -.0600 0621 O135 
in C-2 N= 1353 N= 133 N= 153 N= 153 N= 153 N= 153 
sig .861 sig .965 sig .568 sig .461 sig .446 sig .869 
°% Time 270) -.0357 O25 _ 0010 0035 -.0180 
in C-3 N= 153 N= 153 N= 153 Na De N= 153 N=133 
sig .74] Sig .661 sig .756 sig .990 Sig .966 SIG 2622 
Poe lime -.089 1 -.0992 Priel _. 1040 _.0247 . 1002 
in C-4 = 153 mn = 153 N= 133 N= 153 N= 153 N= 153 
sig .273 Sig eeee SI NM SKe sig .201 sig ./62 sig .218 
3M -.2093 1940 -.0321 1904 -. 1903 0299 
n— 92 = 98 N= 98 = i 3 N= 98 
sig .039 sig .056 sig ./54 sig .060 sig .OO1 sig ./70 
REFTRA Poon -.0914 1019 -.1534 1146 -.1598 
N= _50 N= _30 = 50 N= _90 N= 50 N=_ 50 
1074250 sig 525 sig .481 Sig 287 sig .428 sig .268 
OPRPE -.1043 02h -.0313 0744 0414 0101 
N=_98 NS ls N=_98 Nir 78 N= N=_98 
sig .307 sig .698 sig ./59 sig .407 sig .685 Sig woz 
PS ie term -.0802 wale 4 mL 0029 -. 1691 -.0640 
Retention N= 14) N= 141 N= 14] N= 141 N= 141 N= 141 
sig .344 sig .O41 sig .887 Sie sig .045 sig .451 
2) ten 0005 5 OS Zz P.O1G2 -.0617 -.0533 -.1169 
Retention m= 139 IN. = 139 N= 139 N= 139 N= 139 N= 129 
sig .995 sig .336 sig .849 sig .470 sig .533 sig .170 
Gareer aa 0096 _.0420 - E0217 -.0522 
Retention N= 139 N= 139 N= 139 N= 139 N= 139 N= 139 
Sige?) sig .910 sig .624 Sig sig .800 sig .942 


35 


vr. since 
Cvie 


Average 
Both 


vr. since 
eet 


Average 
Officers 


Average 
Vr since 
viet 

Enlisted 


TABLE 3 
SPEARMAN’S RHO RANK CORRELATION COEFFICIENTS 
PACIFIC FLEET 
Percent 
Both 
w, LMET 


ee viel 


Percent 
Emlisted 
we Vi Ed 


3M 


ON OO 


m= CONIC 


CELE 
Fomtenin 
Retention 
Dw tenn 
Retention 
Career 
Retention 
SMI 


allow the researcher to reject the null hypothesis (that the two variables are unrelated) 
and conclude that the two variables are related. 

Once it appears that the two variables are related to each other, the sign (- or +) 
of p, should be reexamined. This is the point at which extreme caution should be 
taken with regard to interpretation. Many of the variables indicate poorer performance 
as their values increase. For example, “percent time in C-4 (not combat ready)” - as it 
increases, readiness of the ship decreases; “average years since attending LMET” - as 


this value increases, the LMET training (if any) of personnel onboard is “rustier”. 


A. ATLANTIC FLEET RESULTS 

Only two of the measures of effectiveness have a significant correlation (at the 
a=.05 level) with anv of the LMET variables. These are 3M scores and first term 
fetemulom erates. Ime 3.Vi scores show a relationship to the percentage of LMET 
graduates, both enlisted and officer, but in opposite directions. Specifically, as shown 
in Table 2, the scores on 3M inspections are negatively related to the percentage of 
enlisted personnel who had attended one or more LMET courses. This is shown by 
the p, of -.2093. However, 3M inspections scores were positively correlated with the 
percentage of officers with LMET training (p, = .1940). 

Interestingly, if one expands consideration to significance at the @ = .10 level, 
this Same trend is noticed in correlations between 3M and average time since attending 
LMET. Here the signs of the coefficients are reversed (positive for enlisted, negative 
for officers) because of the “reverse” nature of the “Average vr since LMET” variable. 
The coefficient for enlisted vears since attending LMET (.1904) indicates that as the 
number of years since attending LMET increases (to a maximum of eight per person), 
the scores on 3M inspections tend to increase. Conversely, the coefficient for officer 
vears since attending LMET (-.1903) indicates that as the average number of years 
since attending LMET increases, the scores on 3M inspections tend to decrease. In 
other words, it appears as though those ships which have, on average, more recent 
enlisted graduates of LMET tend to attain lower scores on 3M inspections; whereas 
ships which have, on average, more recent officer graduates of LMET tend to attain 
higher scores on 3M inspections. 

“First Term Retention” was the only other MOE to correlate significantly with 
any of the LMET variables. The percentage of officers who had attended LMET was 


positively correlated with first term retention rates. Those ships which had higher 
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percentages of officers who had attended at least one LMET course tended to have 
higher rates of retention of first-term personnel. Reinforcing this finding is the 


negative p, between first term retention and average years since attendance for officers. 


B. PACIFIC FLEET RESULTS 

There were more significantly related measures for Pacific Fleet ships than for the 
Atlantic Fleet, however, the results were mixed. Some significant relationships between 
the C-rating indicators of readiness and LMET attendance were found. What the 
results show is that the percent of time ships spent in categories C-1 and C-2 1s 
negatively related to the percent of officers with LMET. To some extent, percent time 
in C-2 1s also negatively correlated with percent of enlisted personnel with LMET 
training. This can be interpreted to mean that as the percentage of officer personnel 
with LMET increases the percent time that the ship is either fully or substantially 
combat ready decreases. This general trend is continued with significant positive 
correlations between percentages of LMET trained officer personnel and percent time 
spent in C-3 and C-4 (marginally and not combat ready). These results are contrary to 
expected improvements in readiness as a result of LMET training of personnel. As one 
might expect “Average yr since LMET - Officers” and”Average yr since LMET - Both” 
were also significantly correlated (with opposite signs due to the “reverse” nature of the 
variables) to C-rating variables (with the exception of “% Time in C-1”). 

Other measures of effectiveness which correlate significantly with LMET 
attendance include first term retention, second term retention and career retention. 
These variables were negatively correlated with “Percent Enlisted w/LMET™” and 
positively correlated with “Average vr since LMET - Enlisted” (although not 
significantly for “First Term Retention”). To interpret this: those ships which had 
higher retention tended to have fewer LMET trained enlisted personnel. Those ships 
with higher retention also had a higher average number of years since their enlisted 
personnel had attended LMET (this average is also affected by setting the number of 


years since attendance to eight for people who had never attended). 


C. SUMMARY 
As shown in Tables 1 and 2, the majority of “cells” did not indicate a significant 
relationship between MOE and LMET variables. In fact, correlations which did 


appear significant Were not present in both fleets. Many of the significant correlations 
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Were counterintuitive and so deserve closer scrutiny. Results were, at best, mixed and 
did not point to any strong relationships between LMET and fleet effectiveness: 
although weak relationships were shown for some measures--usually on the order of p, 


= .19 to .27. These results can lead to tempered conclusions and several suggestions 


for improved research in the area of LMET evaluation and fleet measures of 


effectiveness. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 

There is a measurable relationship between LMET and several fleet MOEs, 
however, this relationship was not consistent in both fleets and was mixed or 
counterintultive in some cases. For the Atlantic Fleet, only two measures had 
significant relationships to any LMET variables--3M inspection scores and first term 
retention. The relationships were opposite for officer and enlisted LMET, the latter 
having a counterintuitive relationship with 3M. 

For the Pacific Fleet ships, there were many more significant relationships, but 
they Were primarily in two areas: C-ratings and retention. Once again the results were 
often the opposite expected--the more time the ship spent in C-1 or C-2 (combat ready 
or substantially combat ready), the fewer personnel had attended LMET. Officer 
LMET did correlate positively with career retention indicating some benefit from 
Evie 

Why were the relationships between LMET and fleet effectiveness scant and in 
some respects, counterintuitive? There are several possibilities: 


e The data on LMET attendance may not be accurate. The system for reporting 
course attendance to NITRAS contains a number of “holes” and “bugs” leading 
to a reputation for unreliable data, especially with regard to older data (LMET 
records were scanned as far back as 1978, when the course was first taught). 


e The data on measures of effectiveness lack clear reliability and validitv. High 
Vear to vear variability and instability of ship performance measures were found 
by Chatfield and Morrison of NPRDC [Ref. 30]. Some measures used could be 
considered parochial (OPPE, SMI) and thus were not truly indicative of 
shipwide performance. 


e LMET may not be having the desired effect on attendees’ subsequent 
performance. It may in fact be counterproductive. 


e LMET may sensitize graduates to imperfections in the fleet environment 
causing them to be less likely to reenlist (see Chapter V, Table 2). 


¢ Competencies and behaviors learned in LMET may not be reinforced 
(rewarded) in the fleet. Behaviors not at least intermittently rewarded (through 
recognition and approval) tend to extinguish rapidly. No measure of degree of 
command support for LMET could be made. Only the number of people in 
each command with LMET was measured--not the extent to which LMET 
competency use is rewarded and reinforced. As stated by George Eggert in an 
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article on management development, “It is unwise to ‘develop’ behavior in 
training programs that will not be reinforced back on the job” [Ref. 33] 


e Selection bias regarding who attends LMET (especially TAD) may be occuring. 
Supervisors and department heads might tend to send those whom the ship can 
best afford to lose for two weeks rather than reward their best performeres with 
two weeks of leadership training. LMET may also be given to higher 
propertions of junior versus senior supervisors. 


e Sending people TAD t> LMET may leave those ships who send more personnel 
shorthanded, temporarily compromising readiness. 


e The differences in the results for the Pacific and Atlantic Fleets may reflect 
Variation in: 
* inspection standards and differences in inspection teams 


* 
* 


Operation schedules and missions 
* frequency of inspections, drills, and distinguished visitors 

* reporting methods and criteria 

* Fleet “climate” (prevailing attitudes and priorities) 

sie e Me Sites mimsinuic cons ete, (East “su WestCoast) 

Though essentially speculative, the possible reasons behind the trends in the data 
point to a need for further evaluation. Not onlv should the method and content of 
LMET be examined but also the methods by which the Navy measures the 
effectiveness of its fleets. 

The ultimate standard that must be used in judging the usefulness of any 
Organizational program is whether or not that program 1s making an impact on “the 
bottom line.” For the U.S. Navy, the bottom line is not a profit figure, but something 
called readiness or effectiveness. Measuring this effectiveness 1s something the Navy 
attempts to do in many ways--only a few of which were included in this study. These 
data are collected for a mvriad of purposes among which are: 

e to insure units are meeting minimum standards 

e to aid in operational planning 

e to gauge whether a unit is prepared for deployment or operational assignments 
e to provide input for unit awards 

Often secondary uses of the information, such as input to the Commanding 
Officer's fitness report or unit award nominations, can obstruct the accurate 
recording reporting of data. Inspections often become an end unto themselves. The 
obiective becomes--passing the inspection--instead of maximizing overall effectiveness 


and capability. A CO may delay the transfer of kev supervisors onboard until after a 
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Major inspection and then lose a substantial proportion of his key people just prior to 
deployment. Many ships use their “first teams” for inspection/training drills as much 
as possible allowing other watch sections to lag in their proficiency. 

All of the measures available for this thesis have at least some limitations ranging 
from inconsistent reporting methods, to ship to ship differences in standards, to 
inflation of grades, to “fudged” reports. Even the data regarding LMET attendance are 
regarded as potentially unreliable--probably in the direction of not including everyone 
who had attended LMET (the technicians at NITRAS could not assure the authors 
that all LMET attendance had been recorded for all courses since 1978). 

Some additional considerations and limitations in applying and interpreting the 
results of this study: 


e The technical nature of 3M inspections and OPPEs--these measures are 
influenced by the seniority and skill of personnel in_ certain 
divisions/departments. 


e Wide variation in ship type which in turn means differences in mission, 
operational schedule, homeport, command climate, etc. 


e Variations in ships’ age, maintenance status, and crew mix (senior/junior, 
officer/enlisted) 


e No control group--as Davies mentioned, the Navv’s policy is not to 
systematically deny LMET training to any targeted group [Ref. 19] In fact 
Navy policy is to send all personnel headed for sea duty to LMET (see chapter 


I). 


e The results of past studies regarding LMET effects on behavior change 
[Refs. 20,21,22] indicated no systematic relationship between LMET and 
improved leader performance. 


The LMET program has gone unevaluated with regard to its effect on 
Operational unit performance for nearly ten years now. To date (including this study) 
no clue has been found that the training helped a single ship. If nothing else, this 
study should provoke more extensive efforts to evaluate LMET against effectiveness 


measures On a Navywide basis. 


B. RECOMMENDATIONS 


1. Improvements to LMET 
The following recommendations for improvements to LMET are based primarily on 


literature review: 


Continue current efforts to include aspects of the Model for Command 
Effectiveness [Ref. 24] in LMET curricula. This will give the LMET graduate 
more of a “systems” view of an organization--how the components and forces 
within and external to the organization interact and impact upon the overall 
effectiveness of the unit. 


\fake further efforts to tailor the courses to the developmental needs of officers 
and NCQs to the stage (early or mid-career) and to the field (warfare or staff 
community). 


Instructors should be closely screened. They should be volunteers, have 
superior training facilitation skills, and be proven in leadership performance. 
This recommendation may be difficult to implement given the reputation of 
LMET and HRM billets as “not career enhancing”. 


Reexamine the methodology--Does LMET include the best known state of the 
art methods for teaching the objectives of LMET? If time is linuted are 
learning objectives prioritized and the most effort spent on the most 
important; difficult areas. 


Given improvements in LMET delivery and content over the last few vears, 
provide more opportunities for attendance by NCO’s. Officer training-has been. 
systematically included in the transfer,training pipeline (SWOS Basic, SWOS 
Department (ead school sPeO PO School). “So0n.sif ot already, many 
officers will have received LMET at all three levels--not as a redundant course, 
but at appropriate depth and emphasis for each target group. Enlisted 
supervisors in this study received LMET in consistently lower proportions than 
their officer counterparts. 


One promising program is the “mobile” LMET team that can perform the 
training for all levels (in appropriate groups) within a single unit. The teams 
are often able to incorporate unit specific issues into the course. This technique 
may be one of the best methods for achieving “critical mass” [Ref. 23], a 
sufficient proportion of LMET trained supervisors within the unit to assure 
agreement and support of LMET competencies. 


2. Areas for Future Research 


By sampling a smaller number of conimands and collecting data directly from 
the ships, research can be conducted at several levels, e.g., LPO, LCPO, Junior 
Officer, Senior Officer. Also this approach might allow breakouts of the data 
by departments or even work centers and individual ratings. A smaller sample 
might also allow the use of measures in addition to those studied here which are 
onlv available at the unit level such as departmental drill performance, 
underwav hours, and inspection comments, and individual performance data. 


Select a smaller sample of “excellent” and “average” commands based on criteria 
such as those used in the Command Excellence Study (Ref. 24] and perform an 
analysis of variance to determine whether the level of LMET training is 
different between the two groups. 
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e Use other Measures of Effectiveness in a similar study: 
* Monthly Training Reports 
* PQS accomplishment 
* Departmental Excellence Awards 
eS SGRSY, 
* NWT I (Nuclear Weapons Technical Inspections) 


¢ Develop a control group--a group of ships on which no members have had 
LMET; then measure their accomplishments over a period of time compared to 
simular ships with LMET trained personnel. 


e Expand the study to other Navy communities--aviation, submarine, special 
warfare, shore, etc. 


e The authors also agree with and offer several recommendations made by 
Chatfield and Morrison regarding Navy MOEs: 


* Commit resources for improving measures of readiness and develop new, 
carefully constructed indices of ship readiness. 


* Measures should assess true operational readiness rather than 
administrative procedures and equipment status. 


* Assessments should be perceived to contribute to the capability of the unit 
and not as inspections for inspection’s sake. 


* Centralize and automate all readiness rating recording and analysis. 
Eliminate redundancy and weight measures according to their importance 
tontne teen 


* Use only external assessment teams or individuals. 
* “Surprise” inspections should be random and live up to their name. 


Standardize the content, administration, and analysis of assessments. 
(Ref. 30] 


e Leadership training should be included in studies using multivariate models to 
predict organizational effectiveness and productivity. 


e Experimentation regarding what the optimum form of LMET 1s: maximum 
benefit to cost ratio, optimal career points for training, optimum course length 
for each career point. 


¢ Implement surveys of TAD graduates’ COs at all LMET sites (see Chapter III, 
LMET sites) 


C. SUMMARY 
The Navy, like other military organizations, faces complex leadership problems 
and therefore complex training issues. Leaders must be capable of converting their 


methods and styles to combat situations as quickly as the need arises. Navy 
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supervisors must Cope with a variety of challenges including increased technology: 
smarter, more capable, but fewer sailors; and fast paced changes in the operating 
environment. Leadership and management tasks are occupying more than half the 
time of most personnel above the grade of E-6. Shouldn't leadership and management 
training be given a higher priority given its relative domination of NCOs’ and Officers’ 
Gemae? 

Joseph Olmstead, in a 1980 report on leadership training, said this in conclusion 


about leadership training: 


Without a doubt, the quality of available leadership at all levels determines the 
character of an organization and the effectiveness with which it accomplishes its 
oojectives. Accordingly, the development of individuals who occupy leadership 
positions is one of the most critical functions in any organization. 

Although difficult when conducted properly, effective training for leadership 1s 
feasible. Despite the fact that the field 1s in a state of disarray and many 
programms are not very effective, there ts sufficient evidence to conclude that 
leadership can be taught when training 1s sincerely deemed important by 
managements and when it is thoughtfully designed and carefully implemented. 
(Ref. 35] 
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APPENDIX A 
VARIABLE LIST 


LMET Variables Mean* Minimum Maximum N 
Yo emiisted wale hte 42 21 66 285 
% Officers w/ LMET 79 SS 100 285 
70 DO le iede 48 28 65 285 
Average years since 5:0 -| 7 285 
attending LMET - Enlisted 

Average years since 3.5 l 6 285 
attending LMET - Officers 

Average vears since 521 4 7 285 
attending LMET - Both 

NIOE Variables Mean* Minimum Maximum N 
Percent time mne-) me oe bes 285 
Percent time in C-2 <— i oe 285 
Percent time in C-3 2x “i ee 285 
Percent time in C-4 = “* 285 
3M Inspection Scores 87 65 98 220 
Refresher Training Scores 1.2 0.0 2.0 [oz 
Operational Propulsion 63 0.0 240 122 
Plant Exam Results 

Supply Management aR 1.0 4.0 123 
Inspection Results 

First Term Retention 50 00 1.00 2a2 
Second Term Retention .68 .O0 1. 33"=* 240 
Career Retention 78 00 L104" 269 


*The reader should be cautioned that mean values for variables that are either already 
expressed as an average or proportion/percentage can not be interpreted as the /fleer 
average for that variable, but rather the arithmetic mean of the entries for all ships. 


** Withheld due to classification of source data 


*** Retention figures greater than one are possible when an individual reenlists prior to 
their “eligibility window” creating a situation where more persons reenlist’extend than 
Were counted as “eligible for reenlistment’. 
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APPENDIX B 


COURSE NUMBERS USED TO DETERMINE LMET ATTENDANCE 


Course Number 


A-012-0037 
A-01 2-0045 
A-4H-0107 
A-4H-O111 
A-4H-0112 
A-3H-O118 
A-500-0033 
A-500-0034 
A-500-0036 
A-/C-0022 
A-7C-0025 
A-8B-09012 

P-00-4302 


Course Name 


Recruit Company Commander 
EVE? Instructor 

SWOS Department Head 
Prospective Commanding Officer 
Feespecuverk ecu. e Ollicer 

By OSsbasic 

Bo viovViGsSixg year Oblicor DMET 
LMET for Leading Pettv Officers 
LMET for Leading Chief Petty Officers 
PME T for Division Officers 

LMET for Aviation Division Officers 
Supply Corps Officer Basic 

Officer Indoctrination School 


* Indicated only for courses that were not strictly LMET courses 
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Yrs in effect” 


1983-1985 
NA 
1978-1982 
1978-1985 
1978-1985 
1978-1985 
NA 
NA 
NA 
NA 
NA 
1978-1985 
1981-1985 


APPENDIX C 


ATTENDANCE FIGURES FOR LMET COURSES 


Course Name Number Attended 


Attended no LMET 26,774 
Recruit Company Commander 200 
LMET Instructor oe 
SWOS Department Head oi 
Prospective Commanding Officer 310 
Prospective Executive Officer 375 
SWOS Basic 3,622 
BT/MM Six Year Obligor LMET S11 
LMET for Leading Petty Officers 14,44] 
LMET for Leading Gel Pemmaetiicers 3,453 
LMET for Division Officers Bee 
LMET for Aviation Division Officers 98 
Supplv Corps Officer Basic 643 
Officer Indoctrination School 76 


% Attended 


51.65 
Bo 
.O7 

1.88 


Of those who attended LMET 88.2 percent attended only one course; 10.2 


percent attended two courses; 1.5 percent attended three; and .1 percent attended four 


Or more courses. 
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APPENDIX D 
DISTRIBUTION OF YEAR OF ATTENDANCE 


Year Frequency Percent 
19%3 40 O72 
1979 505 Z 
L980 1,170 4.8 
198] Deoie on! 
1982 4,913 20.0 
1983 Sy AT oan) 
1984 4,994 20.4 
1985 4.714 te 
Total 24,505 100.0 
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